On Error-related Potentials during Sensorimotor-based Brain-Computer Interface: Explorations with a Pseudo-Online Brain-Controlled Speller

Objective: Brain-computer interface (BCI) spelling is a promising communication solution for people in paralysis. Currently, BCIs suffer from imperfect decoding accuracy which calls for methods to handle spelling mistakes. Detecting error-related potentials (ErrPs) has been early identified as a potential remedy. Nevertheless, few works have studied the elicitation of ErrPs during engagement with other BCI tasks, especially when BCI feedback is provided continuously. Here, we test the possibility of correcting errors during pseudo-online Motor Imagery (MI) BCI spelling through ErrPs, and investigate whether BCI feedback hinders their generation. Results: Ten subjects performed a series of MI spelling tasks with and without observing BCI feedback. The average pseudo-online ErrP detection accuracy was found to be significantly above the chance level in both conditions and did not significantly differ between the two (74% with, and 78% without feedback). Conclusions: Our results support the possibility to detect ErrPs during MI-BCI spelling and suggest the absence of any BCI feedback-related interference

Brain-Machine Interfaces: A Tale of Two Learners

Brain-machine interface (BMI) technology has rapidly matured over the last two decades, mainly thanks to the introduction of artificial intelligence (AI) methods, in particular, machine-learning algorithms. Yet, the need for subjects to learn to modulate their brain activity is a key component of successful BMI control. Blending machine and subject learning, or mutual learning, is widely acknowledged in the BMI field. Nevertheless, we posit that current research trends are heavily biased toward the machine-learning side of BMI training. In this article, we take a critical view of the relevant literature, and our own previous work, to identify the key issues for more effective mutual-learning schemes in translational BMIs that are specifically tailored to promote subject learning. We identify the main caveats in the literature on subject learning in BMI, in particular, the lack of longitudinal studies involving end users and shortcomings in quantifying subject learning, and pinpoint critical improvements for future experimental designs

Context–aware Learning for Generative Models

This work studies the class of algorithms for learning with side-information that emerges by extending generative models with embedded context-related variables. Using finite mixture models (FMMs) as the prototypical Bayesian network, we show that maximum-likelihood estimation (MLE) of parameters through expectation-maximization (EM) improves over the regular unsupervised case and can approach the performances of supervised learning, despite the absence of any explicit ground-truth data labeling. By direct application of the missing information principle (MIP), the algorithms' performances are proven to range between the conventional supervised and unsupervised MLE extremities proportionally to the information content of the contextual assistance provided. The acquired benefits regard higher estimation precision, smaller standard errors, faster convergence rates, and improved classification accuracy or regression fitness shown in various scenarios while also highlighting important properties and differences among the outlined situations. Applicability is showcased with three real-world unsupervised classification scenarios employing Gaussian mixture models. Importantly, we exemplify the natural extension of this methodology to any type of generative model by deriving an equivalent context-aware algorithm for variational autoencoders (VAs), thus broadening the spectrum of applicability to unsupervised deep learning with artificial neural networks. The latter is contrasted with a neural-symbolic algorithm exploiting side information

The Cybathlon BCI race: Successful longitudinal mutual learning with two tetraplegic users

This work aims at corroborating the importance and efficacy of mutual learning in motor imagery (MI) brain–computer interface (BCI) by leveraging the insights obtained through our participation in the BCI race of the Cybathlon event. We hypothesized that, contrary to the popular trend of focusing mostly on the machine learning aspects of MI BCI training, a comprehensive mutual learning methodology that reinstates the three learning pillars (at the machine, subject, and application level) as equally significant could lead to a BCI–user symbiotic system able to succeed in real-world scenarios such as the Cybathlon event. Two severely impaired participants with chronic spinal cord injury (SCI), were trained following our mutual learning approach to control their avatar in a virtual BCI race game. The competition outcomes substantiate the effectiveness of this type of training. Most importantly, the present study is one among very few to provide multifaceted evidence on the efficacy of subject learning during BCI training. Learning correlates could be derived at all levels of the interface—application, BCI output, and electroencephalography (EEG) neuroimaging—with two end-users, sufficiently longitudinal evaluation, and, importantly, under real-world and even adverse conditions

Subject-oriented training for motor imagery brain-computer interfaces

Successful operation of motor imagery (MI)-based brain-computer interfaces (BCI) requires mutual adaptation between the human subject and the BCI. Traditional training methods, as well as more recent ones based on co-adaptation, have mainly focused on the machine-learning aspects of BCI training. This work presents a novel co-adaptive training protocol shifting the focus on subject-related performances and the optimal accommodation of the interactions between the two learning agents of the BCI loop. Preliminary results with 8 able-bodied individuals demonstrate that the proposed method has been able to bring 3 naive users into control of a MI BCI within a few runs and to improve the BCI performances of 3 experienced BCI users by an average of 0.36 bits/sec

Post-adaptation effects in a motor imagery brain-computer interface online coadaptive paradigm

Online coadaptive training has been successfully employed to enable people to control motor imagery (MI)-based brain-computer interfaces (BCIs), allowing to completely skip the lengthy and demotivating open-loop calibration stage traditionally applied before closed-loop control. However, practical reasons may often dictate to eventually switch off decoder adaptation and proceed with BCI control under a fixed BCI model, a situation that remains rather unexplored. This work studies the existence and magnitude of potential post-adaptation effects on system performance, subject learning and brain signal modulation stability in a state-of-the-art, coadaptive training regime inspired by a game-like design. The results extracted in a cohort of 20 able-bodied individuals reveal that ceasing classifier adaptation after three runs (approx. 30 min) of a single-session training protocol had no significant impact on any of the examined BCI control and learning aspects in the remaining two runs (about 20 min) with a fixed classifier. Fifteen individuals achieved accuracies that are better than chance level and allowed them to successfully execute the given task. These findings alleviate a major concern regarding the applicability of coadaptive MI BCI training, thus helping to further establish this training approach and allow full exploitation of its benefits

Evidence Accumulation in asynchronous BCI

The non-invasive Brain-Computer Interface (BCI) developed in our lab targets asynchronous operation of devices by monitoring electroencephalographic (EEG) activity and identifying oscillatory patterns that the user can voluntary modulate through the execution of motor imagery (MI) tasks. Successful self-paced interaction under this framework requires the incorporation of an evidence accumulation module to eliminate the uncertainty of single-sample classification and to drive an efficient feedback visualization. In this work, we motivate the need for this additional module, describe its role in a closed-loop MI BCI and present a comparative study of two different frameworks for evidence accumulation

Sensory threshold neuromuscular electrical stimulation fosters motor imagery performance

Motor imagery (MI) has been largely studied as a way to enhance motor learning and to restore motor functions. Although it is agreed that users should emphasize kinesthetic imagery during MI, recordings of MI brain patterns are not sufficiently reliable for many subjects. It has been suggested that the usage of somatosensory feedback would be more suitable than standardly used visual feedback to enhance MI brain patterns. However, somatosensory feedback should not interfere with the recorded MI brain pattern. In this study we propose a novel feedback modality to guide subjects during MI based on sensory threshold neuromuscular electrical stimulation (St-NMES). St-NMES depolarizes sensory and motor axons without eliciting any muscular contraction. We hypothesize that St-NMES does not induce detectable ERD brain patterns and fosters MI performance. Twelve novice subjects were included in a cross-over design study. We recorded their EEG, comparing St-NMES with visual feedback during MI or resting tasks. We found that St-NMES not only induced significantly larger desynchronization over sensorimotor areas (p<0.05) but also significantly enhanced MI brain connectivity patterns. Moreover, classification accuracy and stability were significantly higher with St-NMES. Importantly, St-NMES alone did not induce detectable artifacts, but rather the changes in the detected patterns were due to an increased MI performance. Our findings indicate that St-NMES is a promising feedback in order to foster MI performance and cold be used for BMI online applications